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Abstract 

This article describes research on the quality of a computer program designed to help secondary 
level science teachers in southern Africa create exemplary paper-based lesson materials. Results of 
this study show that the content, support, and interface of the program combine to form a tool 
that is considered by both users and experts to be valid and practical. Findings further indicate 
that it has the potential to help users create good quality materiab while learning from the 
design process, but that this potential depends primarily on how the program is implemented. 
(Keywords: ICT, curriculum development, teacher professional development.) 

Within the last few decades, many southern African countries have achieved 
independence. Among other changes, this has often brought about new curri- 
cula for basic education, new subject syllabi, and reform with regard to teaching 
methods (such as a call for more learner-centered teaching). Although post-in- 
dependence reform efforts carry promise and optimism, they require immense 
investments with regard to the development of expertise in order to reach frui- 
tion. To complicate matters, many countries in this region of the world have 
to make due with an un(der) qualified teaching force (Caillods, Gottelmann- 
Duret, & Lewin, 1996). This already grave problem can be compounded when 
educational change demands the inclusion of subject matter completely new to 
practicing teachers and/or the adoption of a new (usually unfamiliar) teaching 
methodology. This is often the case when more equitable access to education is 
part of the reform agenda. For example, when Namibia gained independence in 
1995, non-white teachers — many of whom were not permitted to learn about 
science under the South African apartheid regime — were suddenly required to 
teach it. Furthermore, they were to do so in a learner-centered fashion, a notion 
totally foreign to most teachers. Although there is little argument regarding the 
inherent value of developments such as this, the resulting challenges and prob- 
lems presented by these changes must be carefully addressed if there is to be any 
chance for successful implementation (cf. Ottevanger, 2001). 

One useful approach to helping teachers cope with large-scale curriculum re- 
form is the design and sharing of teaching materials among regional or national 
colleagues (Gray, 1998; Ottevanger, 2001; Van den Akker, 1998). The process 
of developing materials can be supported in many ways, including through the 
use of information and communications technologies (ICT). Previous research 
on the added value of ICT during curriculum development has produced 
promising results (Nieveen, 1997), particularly with regard to the creation of 
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classroom materials (Nieveen & van den Akker, 1999). These findings, together 
with recommendations from the field, provided the impetus for the creation of 
a computer program called CASCADE-SEA (Computer Assisted Curriculum 
Analysis, Design and Evaluation for Science [and mathematics] Education in 
Africa). By guiding users through an iterative process of analysis, design and 
evaluation, CASCADE-SEA helps secondary level science and mathematics 
teachers create paper-based teacher guides. Because these teacher guides offer 
commendable examples of how certain curricular innovations can be carried out 
in practice, they are referred to as exemplary materials. These exemplary materi- 
als are usually photocopied and shared among colleagues. 

Extensive research was conducted during the design and development of CAS- 
CADE-SEA. Findings from formative evaluations of prototypes during product 
development shaped the content, support and user interface of CASCADE-SEA 
(McKenney & Van den Akker, 2005). This purpose of this article is to describe 
the summative evaluation of the final version of the CASCADE-SEA software. 

THEORETICAL FOUNDATIONS 

The CASCADE-SEA program is based on the notions that teacher profes- 
sional development and curriculum development can occur simultaneously dur- 
ing the process of materials development, and that ICT can offer added value 
in facilitating that process. Used in a setting where the need for materials is 
great, this software offers educators the opportunity to learn about curriculum 
development and reflect on their own ideas about teaching and learning while 
designing much-needed, good quality teaching materials. Theoretical underpin- 
nings for this perspective are presented below. 

Professional Development 

Teacher professional development is considered important for a host of rea- 
sons (cf. Eraut, 1994), ranging from certification to broadening the teaching 
repertoire (Hopkins, 1998) to improving pupil achievement (Van Blanken, 
1995). And continued professional growth of teachers is widely accepted as an 
essential ingredient to any educational reform (Black & Atkin, 1996; Fullan, 
1991; Loucks-Horsley, Hewson, Love, & Stiles, 1998; Van den Akker, 1996; 
Yager, 1994). The CASCADE-SEA program was designed specifically for fa- 
cilitator teachers, often working in Teacher Resource Centers (TRCs), whose 
inservice agenda includes guiding teams of teachers in the preparation of exem- 
plary lesson materials. This includes materials created to help teachers imple- 
ment curriculum changes or simply to begin to fill a profound void of teaching 
resources. CASCADE-SEA was designed to be consistent with professional 
development experiences that exemplify seven principles identified by Loucks- 
Horsley et al. (1998), namely, those that: 

• are driven by a well-defined image of effective classroom learning and 
teaching 

• provide opportunities for teachers to build their knowledge and skills 

• use or model with teachers the strategies they will use with their students 

• build a learning community 
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• support teachers to serve in leadership roles 

• provide links to other parts of the educational system, and 

• are continuously assessing themselves and making improvements to 
insure positive effect on teacher effectiveness, student learning, leader- 
ship, and the school community. 

Putnam and Borko (2000) give attention to the importance of situated learn- 
ing opportunities in terms of teacher development and curriculum develop- 
ment. They identify a common problem, saying that both novice and experi- 
enced teachers often complain that learning experiences outside the classroom 
are too removed from the day-to-day work of teaching to have a meaningful 
effect. They identify certain beneficial contexts within which teachers’ learning 
might be meaningfully situated: their own classrooms, group settings where 
participants’ teaching is the focus of discussion, and settings emphasizing teach- 
ers’ learning of subject matter. Related to each of these contexts, a clear role for 
curriculum materials (creation) can be identified. 

Learning by Design 

According to Jonassen and Reeves ( 1 996), “ . . . the people who seem to learn 
the most from the systematic instructional design of instructional materials are 
the designers themselves” (p. 695). Other instructional designers (Paquette, Au- 
bin, & Crevier, 1994) emphasize the learning opportunities presented by design 
activities, as do experts in the area of curriculum change in developing coun- 
tries (Dove, 1986; Montero-Sieburth, 1992). It therefore comes as no surprise 
that many inservice educators advocate involvement of teachers in curriculum 
development as an effective form of professional development (Ball & Cohen, 
1996; Ben-Peretz, 1990; De Feiter, Vonk, & Van den Akker, 1995; Dlamini, 
Putsoa, Campbell & Lubben, 1996). For example, Ben-Peretz (1990) advocates 
teacher participation in curriculum development because of the opportunities it 
provides for experiencing decision making with regard to content, instructional 
strategies, scope, and sequence. McKenney (1995) found that the learning 
gleaned from creating, as well as using, good quality materials in Namibia can 
serve to bolster confidence in teaching and, as a result, improve overall class- 
room performance. Still others have found that products developed by practic- 
ing teachers have more credibility to other teachers as being something that 
is truly usable in the classroom (Doyle & Ponder, 1978; Viggiano & Dixon, 
1998). Gray (1998) notes additional benefits of involving teachers in exemplary 
materials design within the context of secondary level science education in 
southern Africa. He found that teachers particularly developed: 

• critical skills with respect to curriculum development 

• writing skills 

• greater confidence in subject-matter knowledge 

• greater professional confidence and morale-boosting, and 

• clarity of their perceptions on teaching and their own classroom practice. 

Although learning and reflection are often promulgated by the development 

of exemplary lesson materials, these processes can also be stimulated by the use 
of them; this notion is addressed in the following section. 
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Curriculum Materials in Developing Countries 

Lockheed and Levin (1993) found that, in developing countries, teacher 
guides that are well integrated with the textbook or other instructional materi- 
als can have a positive effect on student achievement. Along with calls for local 
and aid organizations to invest in the development of locally relevant classroom 
materials (Clegg & Osaki, 1998; Hawes, Coombe, & Lillis, 1986; Lockheed & 
Levin, 1993; Williams, 1986), international implementation research has ex- 
plored the role of exemplary materials in supporting curriculum change. Exem- 
plary materials, such as teacher guides, have been found to be especially useful 
during the initial phases of curriculum implementation. Van den Akker (1998) 
summarizes three main advantages offered by exemplary materials: 

• Clearer understanding of how to translate curriculum ideas into class- 
room practice 

• Concrete foothold for execution of lessons that resemble the original 
intentions of the designers 

• Stimulation of reflection on one’s own role with the eventual possibil- 
ity of adjusting one’s own attitude toward the innovation. 

To fully realize the potential benefits of exemplary lesson materials, an impor- 
tant question then arises: What kind(s) of materials are most useful? 

Characteristics of Materials 

Research has shown that teacher support materials including step-by step guide- 
lines can offer the aforementioned benefits (Brophy & Alleman, 1991; Clegg & 
Osaki, 1998; De Feiter, Vonk, & Van den Akker, 1995; Ottevanger, 2001; Thijs, 
1999; Van den Akker, 1988). Additional supporting arguments for including highly 
specified procedural guidelines in teacher lesson materials are offered by Hameyer & 
Loucks-Horsley (1989). They argue that the success of innovative efforts “depends 
considerably on the quality and demandingness of the materials” (p. 14). Especially 
when curriculum change is on the agenda, they recommend the following: 

Any new product, according to recent research, should be suf- 
ficiently flexible for varied use, applicable to different schemes 
of teaching, its fundamental aims clearly exposed and the con- 
ditions under which it works clearly specified. These features 
do not diminish the necessity of making the indispensable 
core components explicit, so to speak the heart of the new. 

Developing exemplary materials is one way to show how the 
innovation might work. As long as a new idea is explained 
only in the shape of a general scheme, it remains insufficient 
for further application. The more complex an innovation is, 
the more necessary is its specificity in terms of materials or 
other products so that the users can understand the new. The 
central point here is the level of specificity, (p. 14) 

Although benefits of exemplary materials have been identified by many other 
researchers (Ball & Cohen, 1996; Roes, 1997; Thijs, 1999; Van den Berg, 1996), 
most of these experts also agree that offering lesson materials alone yields limited 
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results. Instead, they recommend a combination of exemplary lesson materials 
along with additional forms of teacher support, such as inservice education (De 
Feiter, Vonk & Van den Akker, 1995; Ottevanger & van den Akker, 1998; Van 
den Akker, 1998). In some cases, materials are introduced to teachers during in- 
service activities; they may also offer lasting support as follow-up after workshops. 

Computer-Supported Curriculum Development 

The CASCADE-SEA program was designed to help teachers create good 
quality materials, possessing the high degree of specificity described in the pre- 
vious section. At the same time, the program was structured to facilitate the 
professional development of its users by providing support to better understand 
and visualize the curriculum development process. This section describes core 
ideas pertaining to the support offered in CASCADE-SEA. 

During the last 1 5 years, many tools have been developed for the purpose of 
supporting the complex process of curriculum development (Grabinger, Jonas- 
sen, & Wilson, 1992; Gustafson & Reeves, 1990; Nieveen, 1997; Nieveen & 
Gustafson, 1999; Rosendaal & Schrijvers, 1990; Wilson & Jonassen, 1991; 
Zhongmin & Merril, 1991). Both developers of various support systems for 
curriculum development as well as advocates of the concept of Electronic Per- 
formance Support System (EPSS) presume several advantages of providing 
computer support. First, it is assumed that the use of these systems will lead to 
an improvement in task performance. According to Gery (1991), people learn 
to perform their tasks more efficiently with an EPSS than in a traditional train- 
ing situation. Because an EPSS can provide advice, information, and instruc- 
tion immediately, or “just in time,” users do not need to remember all issues 
related to their work, but they can consult the EPSS regarding the issue they 
want at the time they really need it (Collis, 1994; Nieveen, 1997). This support 
can thereby reduce the information load during task performance, and perhaps 
even increase one’s ability to focus on isolated aspects when necessary. An EPSS 
for curriculum development can encourage a more structured approach and 
further the internal consistency of design decisions (Gustafson & Reeves, 1990; 
Nieveen, 1997). Secondly, the use of EPSSs can help promote organizational 
learning by capturing and sharing knowledge and conventions (Stevens & 
Stevens, 1995). Finally, the use of a computer support system can help forge a 
common language among users and, as a result, increase the quality of commu- 
nication (Flechsig, 1989). 

In discussing the potential benefits of exemplary lesson materials, it was previ- 
ously noted that materials on their own stand to yield far less effect than when 
embedded in a larger framework such as a teacher inservice program. Similarly, 
CASCADE-SEA was designed to dovetail with the ongoing activities atTRCs. 
The support in the program primarily aims to help users improve their task per- 
formance — i.e., create better quality materials than they otherwise would. The 
structure offered within the system also helps facilitate a shared vision of the 
curriculum development process and a common language with which to discuss 
it. Additional support aspects are addressed in the program description below. 
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CASCADE-SEA: AN OVERVIEW 

The CASCADE-SEA program aims to support those groups and individuals 
involved in the process of creating exemplary lesson materials or teacher guides, 
usually to be shared among colleagues in the same region. The procedural and 
conceptual model for curriculum development that is supported within this 
program is presented in Figure 1 . 

This model may be classified as cyclic (emphasizing an iterative approach) as 
well as organic (explicitly featuring the core ideas driving the innovation). As 
this model depicts, four main phases of curriculum development are discerned: 
rationale, analysis, design, and evaluation. The rationale contains the main 
aims and ideas behind the yet-to-be-developed curriculum materials, including 
considerations based on the target setting. Located at the hub of the process, 
the rationale influences all other phases. The process of defining a rationale is 
likely to raise additional questions. In many cases, the best way to answer such 
questions is to conduct a needs and/or context analysis. An elaborated rationale, 
captured in a “profile,” provides the curriculum developers with guidelines dur- 
ing the design phase and may also serve as criteria against which (formative) 
evaluation can take place. Due to the detailed nature of the program, (includ- 
ing approximately 250 different activity screens), the following description of 
the four main components is quite limited. For additional detail, please refer to 
McKenney (2001) or visit the CASCADE-SEA research Web site at http://proj- 
ects.edte.utwente.nl/cascade/seastudy/. 



Figure 1. CASCADE-SEA: Procedural and conceptual model. 
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Rationale 

This portion of the program has been designed to assist the users in reflecting on 
their own ideas by focusing on the questions of who, what, where, when, how, and 
why as they pertain to the (to be created) exemplary lesson materials. This portion of 
the program serves two purposes simultaneously. First, it stimulates users to reflect on 
their own (often not-as-yet articulated) ideas with regard to the creation of good qual- 
ity lesson materials. As discussed above, such reflection promotes professional develop- 
ment while simultaneously improving the quality of the curriculum being developed. 
Second, throughout this portion of the program, the computer is gathering informa- 
tion about the users. This information will later be used (in the analysis, design and 
evaluation components) to provide tailor-made advice, based on the users own situa- 
tion. The four sections of the rationale component are mapped out in Figure 2. 

Analysis 

This component of the program helps users — usually facilitator teachers, 
working in small teams making lesson plans for larger groups of teachers within 
their region — learn more about the needs and the context of the people who 
will use their lesson materials. It stimulates such consideration to increase the 
chance that the materials made will actually be beneficial to other teachers. The 
eight sections of the analysis component are shown in Figure 3 (page 174). 


Design 

For those users who have an idea of what they would like to create, this component 
of the system offers support in designing and building paper-based lesson materials. 

It encourages (but does not require) users to build a lesson series, in the form of a 
teacher guide. Users are free to choose between more and less structured designing 
approaches. Figure 4 (page 174) depicts the six sections of the design component. 
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Figure 2. CASCADE-SEA: Rationale component map. 
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Figure 3. CASCADE-SEA: Analysis component map 
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Figure 4. CASCADE-SEA: Design component map. 

Evaluation 

In this component, information about drafted materials is collected, analyzed, 
and interpreted in a systematic manner, with the goal of determining how they 
may be improved. As illustrated in Figure 5, there are eight sections in the 
evaluation component. It bears a significant resemblance to the structure of the 
analysis component, due to the fact that both of these phases feature research 
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Do the materials work as I had hoped? 
How can they be improved? 
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Figure 5. CASCADE-SEA: Evaluation component map. 


activities. The main difference between these two components pertains to the 
questions asked, and the strategies used to answer them. 

The program also contains a Windows help package, how-to-use tutorials and 
an interactive agent offering context-sensitive guidance. Figures 6 and 7 (page 
176) help convey the look and feel of the program. Figure 6 shows the database 
link during the design of a lesson (design component). Figure 7 illustrates how 
the computer offers support during the instrumentation section of evaluation. 

RESEARCH QUESTIONS 

To inform future implementation efforts, the primary goal of the final evalu- 
ation was to determine if use of the CASCADE-SEA program would actually 
yield the desired benefits of helping to produce good quality materials while 
facilitating teacher professional development in the process. Because this study 
took place during a relatively brief period of six months, genuine long-term 
effects of use could not be examined. So rather than attempting to collect con- 
clusive evidence of system effectiveness, data were collected to explore the likely 
value of implementing this system on a full scale. To help understand and inter- 
pret those findings, a secondary goal was to assess the validity of the program’s 
substance and its and practicality of use. If a product is known to be invalid or 
impractical, the chances of it yielding the desired effect can be safely estimated 
as low. Therefore, validity and practicality are seen as prerequisites for bearing 
effect potential. The main research question guiding this evaluation was: What 
are the validity, practicality, and effect potential of CASCADE-SEA? 

The concepts of validity, practicality, and effect potential were further elabo- 
rated. A product is considered valid when it contains state-of-the-art knoivledge 
that is relevant to the tasks it intends to support, and is offered in an internally 
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consistent fashion. Practicality relates to the way a tool “fits” with the target set- 
ting. Practicality includes the notions of instrumentality (usability); congruence 
with user needs, wishes, attitudes, and beliefs; and cost effectiveness in terms of 
investments made in time, effort, satisfaction, and learning. The effect potential 
pertains to the likelihood of CASCADE-SEA to meet its ultimate aims of con- 
tributing to the creation of good quality materials and helping users learn from 
the design and development experience {professional development) . Pertaining to 
each of these qualities, data were collected on the content, support, and struc- 
ture of the system or the resulting materials (the latter was the case in assessing 
the quality of products created with the aid of CASCADE-SEA). 

METHODS 

Within the six months available for the final evaluation of the CASCADE- 
SEA program, three cycles of data collection were undertaken to ascertain the 
validity, practicality, and effect potential of this program: two field tryouts and 
one expert appraisal. The tryouts took place in collaboration with target users 
from professional development programs, whereas the expert appraisal involved 
professionals in the areas of: (science) education in developing countries, (sci- 
ence) teacher professional development, and the use of ICT in education. The 
descriptions below contain details regarding the approach, participants, and 
instrumentation for each cycle. 

Tryout 1 : Tanzanian materials writers 

The first tryout in the evaluation of the CASCADE-SEA final product took 
place in collaboration with a science and mathematics education improvement 
program, based at the University of Dar es Salaam’s Faculty of Education. In 
one of their Materials Writing Workshops, where teachers come together for 
a week during school holidays to create teacher guides, CASCADE-SEA was 
used. Because limited computers were available and this was the first time that 
computers would be involved in one of these workshops, it was not feasible for 
everyone to gain in-depth exposure to the program. Instead, a demonstration 
of CASCADE-SEA was given to all participants (n=19) and then five materials 
writers were selected on a volunteer basis to use the system. The small group 
(n=5) worked with the program for five days, with a few breaks for small and 
large group discussions, including plenary reporting sessions in which the CAS- 
CADE-SEA group related their experiences to the other writers. Data were col- 
lected through: (a) recording discussions, (b) observation during the hands-on 
sessions, (c) analysis of documents produced, (d) a general questionnaire for all 
teachers with a small section on their impressions about CASCADE-SEA, and 
(e) a more detailed questionnaire for the CASCADE-SEA user group. 

Tryout 2: Namibian facilitator teachers 

Facilitator teachers from a Teacher Resource Center (TRC) in rural northern 
Namibia used CASCADE-SEA during a course on curriculum materials design 
that was part of a tailor-made training program. The group (n=9) worked with 
CASCADE-SEA throughout the month-long course, wherein opportunities 
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were sought to combine research goals with meaningful activities for the par- 
ticipants. Data were collected through: (a) notation of classroom discussions; 

(b) observation of participants when working hands-on with CASCADE-SEA; 

(c) participant logbooks which were filled after each of the 10 class meetings; 

(d) analysis of documents created by participants (analysis and evaluation plans, 
lesson plans, etc.), in particular the final assignments, which asked for critical 
reflection on the month’s activities; and (e) a questionnaire that was completed 
at the very end of the course. 

Expert Appraisal 

The expert appraisal took the form of a one-day workshop. Following a brief 
introduction, participants (n=17) spent the morning exploring the CASCADE- 
SEA program in a hands-on session. In the afternoon, experts were asked to 
reflect on issues relating to the validity, practicality, and effect potential of the 
CASCADE-SEA program. They wrote their reactions on nine posters. The 
posters contained leading questions pertaining to the validity, practicality and 
effect potential of the programs content, support, and structure. Thereafter, 
three groups were formed: the content group contained primarily curriculum 
development and teacher professional development experts; the support group 
contained experts on education in developing countries, and the structure group 
contained ICT experts. These groups used the comments on the posters (three 
thematic posters were given to each group) as input for their discussions cen- 
tered around two main questions: 

• Does CASCADE-SEA have the potential to yield a positive effect in 
terms of curriculum development and/or teacher professional develop- 
ment? 

• What recommendations can be made regarding further elaboration 
and implementation of this system? 

They later reported back to the whole group in a plenary session. Data were 
collected through: (a) observation during the hands-on session and during the 
small group discussions, (b) analysis of the poster comments, (c) videotaping of 
the plenary discussion, and (d) a questionnaire administered to all participants 
at the end of the day. 

Data Analysis 

Where necessary (e.g. expert appraisal discussion), data were translated be- 
fore being transcribed and summarized. Data were analyzed deductively and 
inductively. First, they were coded according to the validity, practicality, and 
effect potential of the system’s content, support, and structure. Thereafter, data 
were given color codes according to emergent patterns. The questionnaires 
and observations were used to identify general trends and themes, while the 
discussions, document analyses and logbooks helped to deepen understanding 
by examining specific, often more personal insights. Within one overarching 
framework, variation of sources, participants, and context helped to triangulate 
(Krathwohl, 1993; Miles & Huberman, 1994) findings on CASCADE-SEA’s 
validity, practicality, and effect potential, as illustrated in Figure 8. 
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Figure 8. Data sources in the summative evaluation of CASCADE-SEA. 

Legend: SAK=State-of-the-Art Knowledge; INC=Internal Consistency; 
INS=Instrumentality; CO= Congruence; COS=Cost; BQ=Better Quality materi- 
als; EPD=Enhances Professional Development. 

RESULTS 

Validity 

State-of-the-Art Knowledge 

Participants in the tryouts responded quite favorably to questions per- 
taining to the state-of-the-art knowledge contained in CASCADE-SEA. 

In contrast, the experts commented on both the strengths and weaknesses 
of the program’s content. From the expert group, frequently mentioned 
strengths included the systematic, comprehensive, and wide-ranging av- 
enues (for materials development) the program offers and the level of detail 
and “enormous amount” of knowledge, information, and tools contained 
inside the program; the inclusion of information to assist with implemen- 
tation was also noted as a particular strength. However, aspects listed by 
some as strengths were noted by others to be weaknesses, for example, 
“[CASCADE-SEA is] too ‘rich’ for the target group — cut out some essenti- 
alities. . .Africa is in need of guidelines.” 

Participants in the second tryout appreciated the advice on materials de- 
sign, citing the knowledge gained and help offered (e.g. “it made my lesson 
preparation easy”) as the two greatest assets. The expert group also com- 
mented on the “good overview of what you have to think of” but recom- 
mended that the system should offer more support on (re) design; that is, 
designing from existing material. Further, this group suggested that much 
of the support inside CASCADE-SEA is implicit, “... we think that (es- 
pecially where it is meant to be a learning tool) it [the support] should be 
made more explicit.” Finally, an education in developing countries expert 
aptly summarized the variety of opinions discussed during the workshop: 
“The comprehensive generic nature is a strength and a source of weakness. 
There is a bit of a barrier to opening up a massive program if the task at 
hand is very specific. One might tend to use the pen and paper or Word, 
instead.” 
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Both groups also offered feedback with regard to the structure of the program. 
The user group identified certain screens in the program that could be more 
technically elegant (and offered specific recommendations for improvement). 
The expert group offered more general commentary: 

• A technically light version is recommended: one that runs quickly and 
easily (on older computers); 

• A Web-based option is recommended; 

• Explore improvement of multiple-user database sharing and access; 
and 

• Could the interface be more flexible/ adaptable? 

Internal Consistency 

The bulk of the internal consistency data collected during this cycle came 
from the expert workshop. In terms of CASCADE-SEA’s content, a few experts 
indicated that they felt consistency inside the product was weak. They felt that 
the program does not guard against inconsistent answers/ responses, and that the 
differences in the levels addressed are perhaps too broad (saying that goals for a 
lesson series are abstract, while lesson contents are specific). There were mixed 
opinions with regard to the relationship between program components. Many 
experts felt that the link was not present, while others termed it a “loose cou- 
pling, sometimes caused by common sense” without a direct (one-to-one) rela- 
tionship. This group indicated that the interface was consistently designed, but 
said that more use should be made of the visual support for the design process. 

Practicality 

Instrumentality 

Most participants in all three cycles indicated that they felt CASCADE-SEA 
guides the user step-by-step in making materials. Participants in the first tryout 
were able to follow the on-screen instructions, but did not hesitate to ask for 
clarification when desired. It was often mentioned in the expert appraisal and 
the second tryout that CASCADE-SEA suggests steps that might not otherwise 
occur to the user. Most participants found this to be a major strength (saying it 
“broadens your horizons” in the expert appraisal and “You’ll even realize what 
you haven’t think about before,” in the second tryout), but a few experts pointed 
out that this can be distracting. The materials developers in Tanzania worked at 
their own speed, some dividing up tasks and some staying in teams, throughout 
the week. Similarly, the facilitator teachers expressed that CASCADE-SEA “... 
helps guide teachers on what they’re supposed to do though it does not prescribe 
what exactly to do.” However, one teacher expressed frustration at “using other 
ideas, rather than one’s own.” Expert opinions were similarly varied. Whereas 
most participants felt that the program does allow users to work at their own 
pace, mixed feedback was given with regard to personal style. One participant 
said that the program may, in fact, “block” creative ideas, although another par- 
ticipant pointed out that skipping over (undesired) steps is “very simple” to do. 

Self-reporting data indicated that most participants found the program’s sup- 
port to be sufficient, and that they understood the content and procedures in 
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the program. Observation data showed, however, that the majority of partici- 
pants in the second tryout requested additional explanation. The expert group 
commented that some of the texts in the program could be polished, with 
particular attention to length (some were considered too long) and terminology 
that some thought to be “too complicated for the target group.” Finally, with 
regard to the clarity of the structure, it was noted that even those participants 
with few computer skills were able to learn their way around the program fairly 
quickly during the first (shorter) tryout. However, observation data emphasized 
that this program does not speak for itself. Introductory sessions are necessary, 
especially for new computer users. 

Congruence 

All three cycles indicated that CASCADE-SEA links well with user contextual 
realities. In their own ways, each group echoed the sentiment of this participant 
from the second tryout: “CASCADE-SEA is not a stand-alone solution; it is a 
support tool that will be used within the existing procedures of developing les- 
son materials.” Experts generally felt that (with training) the support would be 
relevant and usable; and 16 out of the 19 participants in the fist tryout said that 
they would be interested in using CASCADE-SEA in future materials writing 
workshops. 

A few participants expressed concern about the lack of computers, but the 
lack of ICT expertise was mentioned by even more experts and users. Said one 
user: “Since most schools in our circuit have an acute shortage of materials to 
support instruction, CASCADE-SEA in my view would be an ideal tool. But. . . 
current realities such as teacher computer literacy level, availability of comput- 
ers and experts to train teachers will have to be addressed. . . . Staff development 
in this circuit would be a crucial condition for successful implementation.” Al- 
though this presents an immense challenge, a materials writer in the first tryout 
had been encouraging. “Look at me,” he said “I knew nothing of the computer 
before this week, and now I have made materials on it.” Most participants in 
this circuit did, however, comment that a week was too short. A resource teach- 
er in the second tryout emphasized the importance of the fact that the program 
has been designed with limited resources in mind, saying that a shared resource 
such as this one would ideally be “strategically deployed” on the few available 
computers (at resource centers). 

Cost 

With all groups, opinions were mixed in terms of the amount of time 
necessary to use the program: participant assessments ranged from “most ef- 
ficient” to “time consuming.” The range is likely explained by the program’s 
flexibility of use, an aspect most participants deemed to be quite satisfactory. 
This relates to a discussion that emerged during the first tryout on whether 
the structure in the program was “required” or “suggested.” Participants in 
the second tryout said that CASCADE-SEA offers a “very flexible way” of 
working on materials and that the user is “free to make certain adjustments 
that are more applicable to [one’s] own context or situation.” However, one 
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participant (second tryout) warned, “CASCADE-SEA may make lazy teach- 
ers more lazy as they will think they don’t need to read books for preparation 
anymore.” Experts suggested that novice users need even more structure and 
that coaching, in addition to introductory workshops, would be desirable. In 
contrast, this group also indicated that they found the analysis component 
to be too detailed. Most participants appreciated the database function and 
recommended that support could be extended (“[provide even] more good 
examples”). 

Effect Potential 
Better Quality Materials 

Most participants were satisfied with the structure of the materials produced 
by CASCADE-SEA. This was evidenced by the fact that participants gener- 
ally maintained the pre-formatted layout for their materials (although they 
did add to it) as well as by their comments in this regard, e.g., “they are well 
formatted but they are flexible to use and [one] can adjust [them] to his or her 
preference.” When asked to rate the quality of materials they had created with 
CASCADE-SEA, all of the materials developers in the first tryout indicated ei- 
ther “better than” or “equal to” materials made the usual way. The user group in 
the second tryout consistently commented that the materials they created with 
CASCADE-SEA were of high quality. The majority of other participants also 
implied, but did not always directly so state, that the quality was better than 
the materials they had made without the computer, e.g., “The lessons are by far 
more detailed.” 

When asked for further specification, three out of the five said that materials 
created with CASCADE-SEA were: (a) more up-to-date, (b) more practical, 
and (c) more effective than those made the usual way. Of the other two partici- 
pants, one consistently said that the these factors were dependent on the user 
(thus, neither better nor worse); the other participant did not answer the ques- 
tion relating to effectiveness, but said that materials made with CASCADE- 
SEA would be less up-to-date and less practical than those made the usual way. 
Two teachers felt that the lesson materials created with CASCADE-SEA were 
more internally consistent than those made the usual way, and one said that 
the materials were less consistent than those made the usual way. The other two 
said that there was no difference, because it “Depends how rich the database is” 
and “because lesson materials are up to date regardless [of whether or not CAS- 
CADE-SEA is used].” 

All five developers in the first tryout indicated that the materials created with 
CASCADE-SEA contain clear useful procedural specifications for the teacher. 
Similarly, one participant in the second tryout wrote, “If care is taken when 
introducing the CASCADE-SEA program then it will definitely benefit the 
users and eventually lead to the improvement of teaching, which in turn will 
lead to the desired educational outcomes of the learners.” The experts felt that 
CASCADE-SEA might have the potential to contribute to (more) valid, practi- 
cal and effective materials (e.g., by “[preventing] haphazardness”) but that this 
depends on two things: how the system is used, and the capabilities of the user. 
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Enhances Professional Development 

Participants in all three cycles concurred that CASCADE-SEA has the po- 
tential to contribute to the professional development of its users. Further, they 
agreed that the program helps users to think about materials development in a 
(more) systematic and thorough fashion. For example, when participants in the 
first tryout were asked whether they thought this program could contribute to 
the professional development of its users, all five (of the intense users) answered 
yes. In addition, four out of five indicated that CASCADE-SEA has caused 
them to think in a more systematic way and all five said that it supported them 
to be more thorough or detailed. Participants in the second tryout were also 
generally positive in this regard, although one participant did note that “[CAS- 
CADE-SEA] makes me reluctant to think.” Finally, the “education in develop- 
ing countries” expert sub-group suggested that this program has potential as a 
learning tool for training, for both preservice and inservice education. They also 
stressed that successful implementation would have to offer sufficient time and 
opportunity for reflection. The “curriculum development” expert sub-group 
said, “We don’t think that this program would prevent improvements in the ar- 
eas of curriculum development and professional development, but as to whether 
or not it can make a contribution: that depends on how the system is used.” 

Five out of five participants in the first tryout indicated that they/ their institu- 
tions would like to use CASCADE-SEA in the future. Further, all five of these 
participants indicated that they had learned about creating/adapting materials 
for their own setting with the aid of this program. All five materials develop- 
ers in the first tryout said CASCADE-SEA helps (teams of) users to visualize 
this process of materials development, and one wrote that it “helps to make 
this work more transparent to us and others.” Further, four out of five found 
the program easy to follow, and none indicated that the program was (to any 
extent) complicated, confusing, or frustrating. Participants in the second tryout 
also stated that they had learned about the process of curriculum development 
through CASCADE-SEA’s structure. They listed things that they had learned 
about the process of materials development, such as “...ways I can ... evaluate 
my materials.” The expert group generally felt that the program’s structure can 
help to visualize the process of materials development, but that it is difficult to 
gain an overview of the process as a whole. 

DISCUSSION 
Revisiting the Findings 

The quality aspects of validity, practicality, and effect potential were examined 
in terms of the program’s content, support, and structure. Compared to the 
relative consensus on support and structure issues, participant opinions were 
especially mixed in terms of CASCADE-SEA’s content (it offered too much for 
some, too little for others). This section discusses possible explanations for such 
disparate findings. 

Beauty, they say, is in the eye of the beholder. One likely reason as to why par- 
ticipant opinions varied so much is because their own needs, wishes, and pref- 
erences were also so diverse. For example, the content of the CASCADE-SEA 
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program was judged by some participants to be (at times too) comprehensive 
and by others to be deficient. Related to this, such variation in opinions sparked 
discussions about the merits of two versions of the program, best characterized 
as: CASCADE-pro (comprehensive) and CASCADE-lite (simple). Generally 
speaking, it was found that those who argued for a simpler version of the pro- 
gram seemed more interested in curriculum development task support per se 
than they were in the other aim of the program: fostering the professional devel- 
opment of its users. While some argued that a lite version of the program might 
be more suitable, others disagreed, echoing the sentiments of Eisner (1994) 
who said that “curriculum development done by teachers can and often does 
take form in the creation of materials, but that curriculum development [by 
teachers] more frequently yields no materials but, rather, plans that might be no 
more sketchy than notes” (p. 127). This group understood that the tool was not 
designed for personal lesson planning, and should not try to support that in this 
context. In contrast, those who saw applications for this tool within professional 
development programs generally tended to appreciate the various components 
of the program as offering users room to grow. To this group, fostering insights 
into curricular processes through the structured analysis and evaluation of exist- 
ing and nascent syllabi and materials (as advocated by Ben-Peretz, 1990) was 
a worthy pursuit. However, even some of those who valued the content of the 
program for its “systematic, rich, clear, thought-provoking” and “logical” nature 
identified concerns about dampening creativity or even encouraging laziness. 

Interestingly, those individuals closely associated with the actual implemen- 
tation of teacher professional development programs (particularly preservice 
teacher educators) were less bothered by such concerns and much quicker to 
articulate their own preferences in terms of how they would use the system. 

This group seemed to view CASCADE-SEA as a source of “curricular possibili- 
ties” and a basis for choice and action, as opposed to a vain attempt to render 
the development process “teacher proof” (c.fi, Ben-Peretz, 1990). Such ideas are 
closely connected with the basic notions upon which this study was founded. 
They also say something about the way in which this program should be imple- 
mented: namely, the use of CASCADE-SEA must be undertaken by (groups 
of) individuals who are confident and competent in ensuring that the program 
be used in such a way as to serve the purposes of their own ongoing endeavors. 

It has been said of lesson materials that many judgments can and should be 
made during implementation (c.f., Snyder, Bolin, & Zumwalt, 1992) and that 
the users of materials are ultimately in control of how they will be implemented 
in the curriculum, (Ben-Peretz, 1994; Clandinin & Connelly, 1992; Fullan, 
2000). Critical selection of useful elements of curriculum materials comes more 
naturally to those who are more experienced and confident. In terms of making 
judgments pertaining to the use of CASCADE-SEA software, it would appear 
that the curricular possibilities are more apparent to those for whom curricular 
adaptation is regular practice (in this case: teacher educators). 

In addition to participant backgrounds and interests, another factor influenc- 
ing beholder perspectives is their expectations. A surprising number of partici- 
pants seemed to expect the CASCADE-SEA program to generate automatic, 
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immediate, and complete results (in the form of materials) for them. At the 
same time, they anticipated this would happen without expending any effort 
and without relinquishing any control over the process. Despite the fact that 
such an attitude contends with some of the basic ideas behind the development 
of the system, it also evidences a certain degree of naivete about the potentials 
of ICT and the processes associated with human-computer interaction. 

Another preconceived notion that was frequently encountered contrasts with 
the aforementioned tendency of certain user groups to consider their own pref- 
erences and approaches to using CASCADE-SEA. Specifically, expert partici- 
pant groups seemed to want to give the computer a greater degree of authority 
than user participant groups. Although they were clearly more critical of the 
program (especially concerning the validity of the content), this group showed 
a predisposition to the assumption that the computer was supposed to offer all 
the (“right”) answers, as opposed to helping users identify the best solutions for 
their own situations. 

Limitations of the Study 

The fact that opinions relating to content were more varied than those on 
support and interface issues gives one pause to consider what, if anything, is 
unique about this aspect. Participants’ preconceived notions about curriculum 
design and development knowledge (as opposed to support or structure knowl- 
edge) may have played a role. Because of their professional orientations, partici- 
pants were probably more adept at critically assessing CASCADE-SEA’s content 
than the other aspects of the program. Although explicit efforts were made to 
shape data collection activities in such a way that all participants could provide 
critical, discriminatory feedback, it is possible that the relative lack of criticism 
regarding support and interface aspects could stem from comparative inabilities 
to do so. This poses a potential threat to the validity of the support and inter- 
face data. 

The scope and duration of the study also presented limitations. Although the 
procedures used facilitated deep and rich understanding of participant experi- 
ences, those insights are limited to the 45 participants involved. Perhaps more 
important, they are limited to the three groups of participants and the corre- 
sponding three strategies through which data were collected. Further, although 
the value of increasing the duration of the expert appraisal remains question- 
able, it seems quite likely that longer tryouts would have been more useful. As 
with any educational innovation, the lasting effects of short bursts of activity 
tend to be minimal. Though not feasibe within this study, (multiple) year-long 
tryouts would surely offer better insight into the long-term value of the CAS- 
CADE-SEA program. 

CONCLUSIONS 

Two main conclusions may be drawn about the validity of CASCADE-SEA. 
First, state-of-the-art knowledge is contained in the program, but choices — par- 
ticularly regarding the quantity — are questionable. Second, although very few 
specific inconsistencies in the program were identified, the general consensus 
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was that internal consistency is present, but weak. In terms of practicality, 
CASCADE-SEA clearly offers step-by-step guidance to the user, but in the eyes 
of some participants, perhaps too much. The final evaluation data emphasizes 
that the program should not be used independently, but rather introduced in 
a training setting, along with follow-up support. Further, the program would 
be most practical and effective when used as a shared resource to supplement 
existing activities in materials development. Data indicate that CASCADE-SEA 
connects well with the needs, wishes, and context of the target users, particu- 
larly in terms of content and support. CASCADE-SEA does possess the poten- 
tial to have a positive effect on the performance of its users, but this potential is 
strongly influenced by how the system is implemented and personal character- 
istics of those using it. CASCADE-SEA can help users to create better quality 
materials than they would on their own. The program can be a useful learning 
tool for inservice or preservice education. However, realizing the potential to 
enhance professional development depends on how the system is used. 

How the system is used, in turn, depends on the vision of those deploying it. 
The program was designed to be used in inservice programs at TRCs — structures 
that have “become widely accepted across southern Africa as an essential ingredi- 
ent of a professional support structure for teachers and schools,” (Hoppers, 1998, 
p. 229). As with any innovation, implementing CASCADE-SEA will likely give 
rise to unforeseen challenges. But the findings from this study do offer guidelines 
for implementation that are likely to increase CASCADE-SEAs contributions to 
the quality of materials created and the users’ learning experiences: 

• The CASCADE-SEA program should be used on a sustained basis 
within a professional development program. 

o Inservice teachers should be made aware that this is meant for 
creating exemplary lesson materials, and not designed for personal 
daily planning. 

o Preservice teachers would benefit from using the program in a 
course that emphasizes the process of lesson planning. 

• CASCADE-SEA should be implemented by facilitators who are al- 
ready competent and confident about their own skills in the area of 
curriculum development. 

• To help their teachers experience support and not be tempted or 
threatened to lose their professional autonomy in terms of curricular 
decision-making, facilitators of implementation should: 

o first formulate their own approach, to safeguard against the naive 
tendency to overestimate what the computer can and should do; and 

o support users in developing skills of critical selection regarding the 
curricular opportunities presented in the CASCADE-SEA program. 

Anzalone (1991) predicted an increased recognition for the importance of 
sound instructional design in the creation of curriculum materials in develop- 
ing countries; he further stated that the development of related capacities would 
undoubtedly be aided by computers. International literature has called for fur- 
ther exploration into the role of computers toward creating opportunities for 
teachers to learn through curriculum and instructional design activities (Gra- 
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binger, Jonassen, & Wilson, 1992; Gustafson & Reeves, 1990; Marx, Blumen- 
feld, Krajcik, & Soloway, 1998; Nieveen, 1997; Putnam & Borko, 2000). This 
study speaks to that need by examining CASCADE-SEAs validity, practicality, 
and effect potential. It has shown that the CASCADE-SEA program does have 
the potential to offer added value to curriculum and teacher development en- 
deavors in southern Africa. However, the extent to which that potential is real- 
ized rests firmly in the hands of those shaping its implementation. 
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